Use Windows Authentication on WCF service behind a SSL handler

After my last blog post about using Cert-based Message security for WCF web service, we started to look into using Windows Authentication for a different system that also sits behind a load balancer/SSL handler. Windows Authentication provides a much easier integration option – client side can simply provide a domain user account to be authenticated, where as in Cert-based authentication, each client needs to install a certificate. This increases difficulties for clients to develop against the service and is our motivation to look into utilizing Windows Authentication instead.

With the experience of cert-based authentication, I was pretty sure it wasn’t going to be easy to use Windows Authentication in a load balanced environment. First thing we tried of course is to follow Microsoft’s guide to use wsHttpBinding with Windows Authentication and Message Security, with one different is that our client needs to use Transport security instead of Message because it must use HTTPS.

Like we thought, this setup didn’t work because the service expect to use Message security but the client is using Transport security. We then tried TransportWithMessage credential and some other settings. None of them works. We were stuck on this error message “The HTTP request is unauthorized with client authentication scheme ‘Ntlm’. The authentication header received from the server was ‘Negotiate,NTLM’.“, which unfortunately is one of those error messages that do not make sense.

In the painful process of pursuing truth, we came across some post raising the problem level to the load balancer level, which discouraged us to keep researching. It seemed more reasonable to find an alternative at that point, and we did find out that using BasicHttpBinding with Windows Authentication and TransportCredentialOnly worked in our environment.

Here is our client setup:

<basicHttpBinding>
  <binding name="BasicHttpBinding">
   <security mode="Transport">
    <transport clientCredentialType="Basic" />
   </security>
  </binding>
</basicHttpBinding>

Service setup:

<basicHttpBinding>
  <binding name="BasicHttpEndpointBinding">
    <security mode="TransportCredentialOnly">
      <transport clientCredentialType="Basic"/> 
    </security>
  </binding>
 </basicHttpBinding>

The problem with this is that the credentials of the client is passed in clear text. Although message before the land balancer is protected by HTTPs, still we want a true end-to-end protection on the credentials. So this solution is off the table. We decided to go back to our original plan.

I will just skip to the end of the story because I like magic!… We eventually found a solution that worked in the load balanced environment. Custom binding once again saved the world! I didn’t find any article about this configuration, which makes it more important to share it with everyone.

Client side:

 <customBinding>
  <binding name="customBinding_WindowsAuth">
    <textMessageEncoding>
      <readerQuotas />
    </textMessageEncoding>
    <security authenticationMode="SspiNegotiated"></security>
    <httpsTransport  authenticationScheme="Anonymous"  
      proxyAuthenticationScheme="Anonymous" useDefaultWebProxy="true">
    </httpsTransport>      
  </binding>
</customBinding>

Service side:

<wsHttpBinding>
 <binding name="WsBindingConfigration" >      
   <security mode="Message">
     <message clientCredentialType="Windows" 
     negotiateServiceCredential="true" algorithmSuite="Default" 
     establishSecurityContext="false"/>
   </security>
 </binding>
</wsHttpBinding>